1. Technical Field
This invention relates to testing printer output. More particularly, the invention relates to testing printer output using regression techniques based on display list comparison.
2. Description of the Prior Art
The computer industry is dynamic and experiencing rapid, continual change. Software and hardware systems continue to improve at an accelerating rate, benefiting synergistically from prior advances in technology. Hence, in light of this technology upheaval, coalescing software and hardware design to produce a desired consistent output remains a formidable challenge.
An iterative diagnostic and quality control method known as regression testing is used to compare outputs from various combinations of hardware and software to determine if a change in one element of the system causes an undesirable change in the output of the system. If a change in the output does occur, the identified error may be used by designers to modify their particular hardware or software to hopefully eliminate the undesirable change in the system output. Unfortunately, identifying the source of the cause of the change in the output can be quite difficult, given the dependencies between the innumerable hardware and software elements of a computer system.
Generally, output from a computer system may include either the results of a calculation, a display on a computer video terminal or tangible printed output from a computer printer. Computer printers must be designed to work appropriately with thousands of different software programs. Software will direct a printer to produce images, graphics and text with differing attributes, such as position, color, shape, size and resolution. Frequently, the computer generates an instruction set to send to the printer based upon a pre-defined printer definition language (PDL), such as ADOBE""S POSTSCRIPT developed by Adobe Systems, Inc. of Mountainview, Calif. The printer interprets the PDL commands to generate the desired printed page.
Printer manufacturers continually strive to upgrade their printer systems to provide higher quality printed output. As print resolution increases and the available spectrum of colors that can be printed expands, substantially more data must be transmitted to the printer to produce the desired output. As a result, previously satisfactory methods for ensuring consistent, quality printer output prove to be inadequate.
For example, checksum comparison is one method used for regression testing of software that produces printed output. For example, when a software program generates and communicates a print command to a printer, the computer will produce a bitmap of the intended page. The checksum for the bitmap is determined by adding the value of the bits or pixels within the bitmap to arrive at a total sum for the bitmap of the particular page. The checksum is a number requiring only a few bytes of storage space. Consequently, a checksum may be generated for a known correct page and stored for later comparison.
If any element of the software is changed, the same original commands may be communicated to the new modified version of the software to generate a new bitmap for the particular page. A new checksum for this new bitmap is then calculated and compared against the original checksum for the known correct page. If the checksum of the original page and the new page are equal, it suggests that the newest version of the software has not affected printer output. However, it does not ensure that the printed output is consistent. For example, an image may appear to be printed correctly, but its position on the page may be different. A checksum comparison test would suggest that the new page has not changed, although in fact, it has. Hence, the checksum regression testing method must be accompanied by manual visual comparison of archived known correct reference pages against new pages to ensure that the software changes have not affected printer output. The need for manual visual comparison causes this approach to be extremely time consuming, unwieldy, and inaccurate. Full regression testing for just one printer description language (PDL) interpreter can comprise upwards of 10,000 pages, for both color and monochrome, and for two or three different resolutions. Hence, an individual, or team of individuals, might be required to visually compare as many as 60,000 pages of output for just one PDL interpreter. Not only is this exceptionally time consuming, but it also tends to introduce a substantial amount of human error in the quality control process due to the tedious nature of the evaluation.
Another common method used to regression test software programs that produce printed output is known as bitmap comparison. In this method, the bitmap for a new page is compared against the bitmap for a known correct page. This approach consumes substantial memory since the known correct reference pages must be stored for all software and hardware configurations being tested. However, the bitmap comparison method does provide more information than the checksum comparison method, since the number and value of different pixels in the bitmap may be counted. Nonetheless, this method still is unable to classify different types of objects in the bitmap to more readily identify the source of a problem. Additionally, the bitmap comparison method is computationally intensive and also requires substantial human interpretation to identify the source of an incorrect output. Hence, as printers become more advanced, the bit map comparison method becomes even more inefficient and introduces substantial human error into the quality control process.
In addition to the basic checksum and bitmap comparison regression testing techniques discussed above, the prior art includes slight variations on these fundamental approaches. For example, Ross, Jr., Method For Improving The Dimensional Accuracy of Laser Printers, U.S. Pat. No. 5,815,187 (Sep. 5, 1998), describes a method for improving the printed output from a laser printer when the moving components of the printer wear with use. Print commands for a known test pattern are sent to the printer. The printed output is then compared to the original known correct test pattern. An adjustment calibration factor is then manually determined and input into the graphics controller to account for the wear in the laser printer. Ross""s method is simply a variation of the bitmap comparison method and is limited in use to adjusting horizontal and vertical dimensional inaccuracies in the printed output.
Edmonds et al, Software Verification Apparatus, U.S. Pat. No. 5,022,028 (Jun. 4, 1991) offers only a slight variation on standard regression testing for software programs. A known set of commands used to generate desired output are automatically replicated and input to a newer revised version of a software program. Edmonds does not suggest any more efficient methods for comparing output generated by the software versions and verifying its accuracy.
Bangston, Method And System For Converting Bitmap Data Into Page Definition Language Commands, U.S. Pat. No. 5,659,638 (Aug. 19, 1997) describes a method for iteratively generating page definition language (PDL) commands from bitmaps generated by scanning the pages of a book. The purpose of Bangston""s invention is simply to provide a means to more accurately reproduce printed pages, when the original source for the printing is no longer available. Bangston""s method is simply a reverse implementation of the standard bitmap comparison approach directed to a very narrow purpose.
McKeeman et al, Method And Apparatus For Producing A Software Test System Using Complementary Code To Resolve External Dependencies, U.S. Pat. No. 5,651,111 (Jul. 2, 1997) describes a software debugging system where separate portions of the computer code are first independently tested and debugged. Then, each code xe2x80x9cunitxe2x80x9d is incrementally combined and tested in a similar isolated manner. McKeeman""s invention is directed to identifying software bugs that result from external dependencies between these software units. The system attempts to identify which lines of code are exercised or branches of code are covered during the testing process. It attempts to minimize regression by dividing the software program into smaller pieces where it is hoped that any problems might be more easily identified, rather than testing the entire software program through a plurality of regressions. Additionally, McKeeman""s system is intended to establish a somewhat standardized testing environment to automate some portions of the regression testing process. However, when applied to regression testing for software producing printed output, McKeeman""s system is unlikely to prove effective. Printed output generally requires the cooperation and interaction of the majority of the modules of the software program. McKeeman""s system does not provide guidance for using complementary code to quickly focus on the proximate cause of errors in printed output.
Chen et al, System And Method For Selecting Test Units To Be Re-run In Software Regression Testing, U.S. Pat. No. 5,673,387 (Sep. 7, 1997) suggests a fairly basic selective approach to regression testing. The essence of Chen""s invention is based on the fundamental observation that one need not test portions of a program which have not been changed. Unfortunately, Chen""s method presumes that sections of code are able to function independently of other sections of code. Although the number of regressions may be reduced using Chen""s method for certain types of software programs, when applied to software used to produce printed output, it would likely fail for similar reasons identified in McKeeman""s system above. The generation of output sufficient for identifying print problems usually results from the interaction of entire portions of the software program, rather than independent entities. Although a portion of code may not have been changed, the code portion may be involved in some manner with generation of PDL for production of printed output.
Accordingly, a need exists for a simple system and method for regression testing of software that produces printed output which requires less human intervention and can more efficiently identify the portion of software code that may be the source of any differences between a new page and a known correct page.
The invention provides a system and method for regression testing of software programs that produce printed output. Display lists, which are created by the software program before rendering graphical objects into bitmap form, are saved by the system and compared to known reference display lists created by an original version of the software program. The display lists include information compartmentalized by object type, including images, graphics or text. Consequently, any differences in the display lists can be readily categorized by object type and attribute, accelerating the process of identifying the source and cause of any difference between a known correct reference page and a new page generated by the revised version of the software program. This regression testing compartmentalization expedites the debugging of the software program.
Display list regression testing has several advantages. It produces greater detail than both the checksum comparison method and the bitmap comparison method.
Additionally, display list regression testing requires less data storage capacity than a bitmap comparison method. Further, display list regression testing allows one to test only those portions of the code whose output is known to be affected by any changes in the software. For example, if a vendor provides a different font, separate comparisons may be performed on the text and non-text parts of the printed page. Consequently, full regression testing of the entire output spectrum is not required, minimizing the time required to debug the software program. Additionally, the need for human visual comparison of output is minimized, thereby reducing the level of human error in the quality control process.